Game-theoretic learning and allocations in robust dynamic coalitional games

The problem of allocation in coalitional games with noisy observations and dynamic4 environments is considered. The evolution of the excess is modelled by a stochastic differential5 inclusion involving both deterministic and stochastic uncertainties. The main contribution is a6 set of linear matrix inequality conditions which guarantee that the distance of any solution of the7 stochastic differential inclusions from a predefined target set is second-moment bounded. As a direct8 consequence of the above result we derive stronger conditions still in the form of linear matrix9 inequalities to hold in the entire state space, which guarantee second-moment boundedness. Another10 consequence of the main result are conditions for convergence almost surely to the target set, when the11 Brownian motion vanishes in proximity of the set. As further result we prove convergence conditions12 to the target set of any solution to the stochastic differential equation if the stochastic disturbance13 has bounded support. We illustrate the results on a simulated intelligent mobility scenario involving14 a transport network

On the Combination of Game-Theoretic Learning and Multi Model Adaptive Filters

This paper casts coordination of a team of robots within the framework of game theoretic learning algorithms. In particular a novel variant of fictitious play is proposed, by considering multi-model adaptive filters as a method to estimate other players’ strategies. The proposed algorithm can be used as a coordination mechanism between players when they should take decisions under uncertainty. Each player chooses an action after taking into account the actions of the other players and also the uncertainty. Uncertainty can occur either in terms of noisy observations or various types of other players. In addition, in contrast to other game-theoretic and heuristic algorithms for distributed optimisation, it is not necessary to find the optimal parameters a priori. Various parameter values can be used initially as inputs to different models. Therefore, the resulting decisions will be aggregate results of all the parameter values. Simulations are used to test the performance of the proposed methodology against other game-theoretic learning algorithms.</p

Adaptive forgetting factor fictitious play

It is now well known that decentralised optimisation can be formulated as a potential game, and game-theoretical learning algorithms can be used to find an optimum. One of the most common learning techniques in game theory is fictitious play. However fictitious play is founded on an implicit assumption that opponents' strategies are stationary. We present a novel variation of fictitious play that allows the use of a more realistic model of opponent strategy. It uses a heuristic approach, from the online streaming data literature, to adaptively update the weights assigned to recently observed actions. We compare the results of the proposed algorithm with those of stochastic and geometric fictitious play in a simple strategic form game, a vehicle target assignment game and a disaster management problem. In all the tests the rate of convergence of the proposed algorithm was similar or better than the variations of fictitious play we compared it with. The new algorithm therefore improves the performance of game-theoretical learning in decentralised optimisation

An evolutionary game perspective on quantised consensus in opinion dynamics

Quantised consensus has been used in the context of opinion dynamics. In this context agents interact with their neighbours and they change their opinion according to their interests and the opinions of their neighbours. We consider various quantised consensus models, where agents have different levels of susceptibility to the inputs received from their neighbours. The provided models share similarities with collective decision making models inspired by honeybees and evolutionary games. As first contribution, we develop an evolutionary game-theoretic model that accommodates the different consensus dynamics in a unified framework. As second contribution, we study equilibrium points and extend such study to the symmetric case where the transition probabilities of the evolutionary game dynamics are symmetric. Symmetry is associated with the case of equally favourable options. As third contribution, we study stability of the equilibrium points for the different cases. We corroborate the theoretical results with some simulations to study the outcomes of the various models

Distributionally Robust Optimization

This chapter presents a class of distributionally robust optimization problems in which a decision-maker has to choose an action in an uncertain environment. The decision-maker has a continuous action space and aims to learn her optimal strategy. The true distribution of the uncertainty is unknown to the decision-maker. This chapter provides alternative ways to select a distribution based on empirical observations of the decision-maker. This leads to a distributionally robust optimization problem. Simple algorithms, whose dynamics are inspired from the gradient flows, are proposed to find local optima. The method is extended to a class of optimization problems with orthogonal constraints and coupled constraints over the simplex set and polytopes. The designed dynamics do not use the projection operator and are able to satisfy both upper- and lower-bound constraints. The convergence rate of the algorithm to generalized evolutionarily stable strategy is derived using a mean regret estimate. Illustrative examples are provided

Dynamic opponent modelling in fictitious play

Distributed optimization can be formulated as an n-player coordination game. One of the most common learning techniques in game theory is fictitious play and its variations. However, fictitious play is founded on an implicit assumption that opponents’ strategies are stationary. In this paper we present a new variation of fictitious play in which players predict opponents’ strategy using a particle filter algorithm. This allows us to use a more realistic model of opponent strategy. We used pre-specified opponents’ strategies to examine if our algorithm can efficiently track the strategies. Furthermore, we have used these experiments to examine the impact of different values of our algorithm parameters on the results of strategy tracking. We then compared the results of the proposed algorithm with those of stochastic and geometric fictitious play in three different strategic form games: a potential game and two climbing hill games, one with two players and the other with three players. We also tested our algorithm in two different distributed optimization scenarios, a vehicle-target assignment game and a disaster management problem. Our algorithm converges to the optimum faster than both the competitor algorithms in the strategic form games and the vehicle-target assignment game. Hence by placing a greater computational demand on the individual agents, less communication is required between the agents. In the disaster management scenario we compared the results of particle filter fictitious play with the ones of Matlab's centralized algorithm bintprog and the centralized pre-planning algorithm of (Gelenbe, E. and Timotheou, S. (2008) Random neural networks with synchronized interactions. Neural Comput., 20(9), 2308–2324). In this scenario our algorithm performed better than the pre-planning algorithm in two of the three performance measures we used